ESQL: Expand type compatibility for match function and operator #117555

carlosdelest · 2024-11-26T12:03:50Z

Match function only worked on text and keyword field types.

The match query actually supports other field types:

Boolean
Date and date_nanos
Numeric types (double, integer, long, unsigned_long)
IP
Version

Adding these supported types to match function and operator, so the following queries are valid:

FROM employees | WHERE still_hired:true

FROM network_traffic | WHERE bytes_out:12749081495402663265

Using casting is also possible to refine the type used:

FROM date | WHERE date:"2023-10-23T13:55:01.543"::DATETIME

FROM date_nanos | WHERE nanos:"2023-10-23T13:55:01.543123456Z"::DATE_NANOS

FROM hosts | WHERE ip : "134.45.21.23"::IP

FROM apps | WHERE app_version: "1.2.3"::VERSION

Using a string will always do the right thing, independently of the type used:

FROM employees | WHERE still_hired:"true"

FROM network_traffic | WHERE bytes_out:"12749081495402663265"

FROM date | WHERE date:"2023-10-23T13:55:01.543"

FROM date_nanos | WHERE nanos:"2023-10-23T13:55:01.543123456Z"

FROM hosts | WHERE ip : "134.45.21.23"

FROM apps | WHERE app_version: "1.2.3"

… with the appropriate object

# Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/PushFiltersToSource.java

github-actions · 2024-11-26T12:04:03Z

Documentation preview:

✨ Changed pages

carlosdelest · 2024-11-26T12:04:26Z

x-pack/plugin/esql/qa/testFixtures/src/main/java/org/elasticsearch/xpack/esql/SpecReader.java

@@ -86,7 +86,7 @@ public static List<Object[]> readURLSpec(URL source, Parser parser) throws Excep
                lineNumber++;
            }
            if (testName != null) {
-                throw new IllegalStateException("Read a test without a body at the end of [" + fileName + "].");
+                throw new IllegalStateException("Read a test [" + testName + "] without a body at the end of [" + fileName + "].");


Unrelated to this change, but useful to pinpoint an error in a CSV test

carlosdelest · 2024-11-26T12:05:19Z

...rc/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/FullTextFunction.java

     */
-    public final String queryAsText() {
+    public final Object queryAsObject() {


Match query needs an object for it to work correctly on specific types. Changing the signature of this function to return an object

carlosdelest · 2024-11-26T12:06:01Z

...rc/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/FullTextFunction.java

        Object queryAsObject = query().fold();
        if (queryAsObject instanceof BytesRef bytesRef) {
            return bytesRef.utf8ToString();
+        } else if (query().dataType() == DataType.UNSIGNED_LONG) {
+            return NumericUtils.unsignedLongAsBigInteger((Long) queryAsObject);


Unsigned long needs to be converted to BigInteger for match function to work correctly

carlosdelest · 2024-11-26T12:07:33Z

...in/java/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/PushFiltersToSource.java

@@ -252,8 +251,6 @@ static boolean canPushToSource(Expression exp, LucenePushdownPredicates lucenePu
                && Expressions.foldable(cidrMatch.matches());
        } else if (exp instanceof SpatialRelatesFunction spatial) {
            return canPushSpatialFunctionToSource(spatial, lucenePushdownPredicates);
-        } else if (exp instanceof Match mf) {


These conditions are checked in validate() / type resolution in Match, so we can simplify

I think that this is fine to remove. I'm also not sure that it was ever really necessary? Since it seems that the field and type were already checked in Match even before this change - which is fine, just trying to understand.

It was unnecessary once we added the Validatable interface, where additional checks can be done once the fields have been resolved.

carlosdelest · 2024-11-26T12:08:48Z

...esql/src/test/java/org/elasticsearch/xpack/esql/expression/function/fulltext/MatchTests.java

+            throw new IllegalArgumentException("Unsupported type in tests: " + dataType);
+        }
+        Object value = EsqlTestUtils.randomLiteral(dataType).value();
+        if (value instanceof BytesRef bytesRef) {


It was a bit tricky to get the correct base object from the randomLiteral function, but I wanted to reuse it for calculating random values for specific data types

carlosdelest · 2024-11-27T06:44:49Z

@elasticmachine run buildkite/docs-build-pr

ioanatia · 2024-12-02T08:33:33Z

docs/reference/esql/functions/types/match.asciidoc

+unsigned_long | long | boolean
+unsigned_long | unsigned_long | boolean
+version | keyword | boolean
+version | version | boolean


this looks great - it's exactly what we need - is this the exact table that gets generated when we run the match tests or did you have to do any changes?
I haven't looked closely at how this gets generated, but I assume that because we have the checkParamCompatibility check then we only get the valid combinations here?

This is automatically generated from the MatchTests, based on the parameters provided as part of the parameters() method.

The test generates the positive test cases, and then uses the errorsForCasesWithoutExamples method to add failing tests for the remaining combinations. These failing tests include the tests that will fail the checkParamCompatibility condition.

ioanatia · 2024-12-02T08:39:13Z

...ugin/esql/src/main/java/org/elasticsearch/xpack/esql/expression/function/fulltext/Match.java

+        }
+
+        if (fieldType.isNumeric() && queryType.isNumeric()) {
+            // When doing an unsigned long query, field must be an unsigned long


what happens otherwise?

There's an explicit error returned - see here.

We only throw an error for queries that are unsigned long when the field is NOT an unsigned long.

fang-xing-esql

Thank you @carlosdelest. The normal workflow works pretty well. I have a mixed data type scenario, should we support it or return an error message ? I'll leave it to you to decide how to address it, as I'm not sure how the match query handles mixed data types. Here are the steps to reproduce it:

+ curl -u elastic:password -X DELETE 'localhost:9200/idx*/'
+ curl -u elastic:password -X PUT 'localhost:9200/idx001?pretty' -H 'Content-Type: application/json' '-d
{
  "mappings": {
    "properties": {
      "numericfield": {
        "type": "integer"
      },
      "stringfield": {
        "type": "text"
      }
    }
  }
}
'

+ curl -u elastic:password -X PUT 'localhost:9200/idx002?pretty' -H 'Content-Type: application/json' '-d
{
  "mappings": {
    "properties": {
      "numericfield": {
        "type": "long"
      },
      "stringfield": {
        "type": "keyword"
      }
    }
  }
}
'

+ curl -X PUT 'localhost:9200/idx001/_bulk?refresh&pretty' -H 'Content-Type: application/json' '-d
{"index": {}}
{"numericfield": "1", "stringfield": "idx001"}
'

+ curl -X PUT 'localhost:9200/idx002/_bulk?refresh&pretty' -H 'Content-Type: application/json' '-d
{"index": {}}
{"numericfield": "2", "stringfield": "idx002"}
'

+ curl -u elastic:password -v -X POST 'localhost:9200/_query?format=txt&pretty' -H 'Content-Type: application/json' '-d
{
  "query": "from idx001 | WHERE match(numericfield, 1)"
}
'
 numericfield  |  stringfield  
---------------+---------------
1              |idx001         

empty result
============
+ curl -u elastic:password -v -X POST 'localhost:9200/_query?format=txt&pretty' -H 'Content-Type: application/json' '-d
{
  "query": "from idx002 | WHERE match(numericfield, 1)"
}
'
 numericfield  |  stringfield  
---------------+---------------

empty result
============
+ curl -u elastic:password -v -X POST 'localhost:9200/_query?format=txt&pretty' -H 'Content-Type: application/json' '-d
{
  "query": "from idx001, idx002 | WHERE match(numericfield::int, 1)"
}
'

 numericfield  |  stringfield  
---------------+---------------

fang-xing-esql · 2024-12-02T21:28:10Z

x-pack/plugin/esql/src/main/antlr/EsqlBaseParser.g4

@@ -78,7 +78,7 @@ regexBooleanExpression
    ;

 matchBooleanExpression
-    : fieldExp=qualifiedName COLON queryString=constant
+    : fieldExp=qualifiedName COLON matchQuery=constant (CAST_OP dataType)?


Is my understanding correct that match always works with literal strings no matter what the field data type is? If this is true, adding explicit casting does seem unnecessary, to be honest, I'm confused by : + :: when I saw it the first time.

I couldn't think of a case that explicit casting is needed for match function or operator, please correct me if I'm wrong. Supporting explicit casting in the matchQuery may give people impressions that we support an expression there, however it is not true. If it is not necessary, shall we just keep it as simple as a constant?

I have a weird case, if we don't have to support explicit casting here, we don't have to deal with this.

+ curl -u elastic:password -v -X POST 'localhost:9200/_query?format=txt&pretty' -H 'Content-Type: application/json' '-d { "query": "from sample_data | WHERE match(event.duration, \"725448\"::time_duration)" } ' { "error" : { "root_cause" : [ { "type" : "illegal_argument_exception", "reason" : "argument of [\"725448\"] must be a constant, received [\"725448\"]" } ], "type" : "illegal_argument_exception", "reason" : "argument of [\"725448\"] must be a constant, received [\"725448\"]" }, "status" : 400 }

carlosdelest · 2024-12-03T12:13:48Z

have a mixed data type scenario, should we support it or return an error message ?

I see, this is interesting. The problem lies on the ES field being mapped as a MultiTypeEsField, which in the end is sent to ES itself as the field type ($$numericfield$converted_to$integer), which does not exist in the mapper and thus match cannot be rewritten to the correct query.

We can remove that restriction by checking that we have a MultiTypeEsField when building the query, and thus sending the real field name.

EVAL would be needed to retrieve the actual values, similar to how multi index works now. So for example:

FROM idx001, idx002 
| WHERE match(numericfield::int, \"1\") 
| EVAL num = numericfield::int

would retrieve the following:

 numericfield  |  stringfield  |      num      
---------------+---------------+---------------
null           |null           |1

The alternative is to detect that at the verification level and issue an error stating that match cannot be used for querying fields with different data types.

As I think allowing this is a better option, I'll start working on it - LMK if you think this should not be allowed.

fang-xing-esql · 2024-12-03T14:46:27Z

As I think allowing this is a better option, I'll start working on it - LMK if you think this should not be allowed.

Multi-typed scenarios are tricky, and I hit a problem with it before on my own PR. For the example above, idx001 return 1 record, idx002 returns 0 record, I was expecting 1 record return when querying both indices in the same query(like how the match query behave below), or get an error message says it is not supported.

+ curl -u elastic:password -v -X GET 'localhost:9200/idx001/_search?format=txt&pretty' -H 'Content-Type: application/json' '-d
{
  "query": {
    "match": {
      "numericfield": {
        "query": "1"
      }
    }
  }
}
'

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "idx001",
        "_id" : "ZyT5jJMBEHEx9eyBYWDC",
        "_score" : 1.0,
        "_source" : {
          "numericfield" : "1",
          "stringfield" : "idx001"
        }
      }
    ]
  }
}

+ curl -u elastic:password -v -X GET 'localhost:9200/idx002/_search?format=txt&pretty' -H 'Content-Type: application/json' '-d
{
  "query": {
    "match": {
      "numericfield": {
        "query": "1"
      }
    }
  }
}
'

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

+ curl -u elastic:password -v -X GET 'localhost:9200/idx001,idx002/_search?format=txt&pretty' -H 'Content-Type: application/json' '-d
{
  "query": {
    "match": {
      "numericfield": {
        "query": "1"
      }
    }
  }
}
'

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "idx001",
        "_id" : "ZyT5jJMBEHEx9eyBYWDC",
        "_score" : 1.0,
        "_source" : {
          "numericfield" : "1",
          "stringfield" : "idx001"
        }
      }
    ]
  }
}

… queries with different types

carlosdelest · 2024-12-03T15:19:53Z

@fang-xing-esql , I've tried to add multi-index, multi-type support in 347b3ce .

I've added some CSV tests for that so you can see what it looks like.

LMKWYT!

fang-xing-esql · 2024-12-03T16:45:56Z

...lugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/EsqlExpressionTranslators.java

+            if (field instanceof FieldAttribute fieldAttribute) {
+                String fieldName = fieldAttribute.name();
+                if (fieldAttribute.field() instanceof MultiTypeEsField multiTypeEsField) {
+                    // If we have multiple field types, we allow the query to be done, but getting the underlying field name
+                    fieldName = multiTypeEsField.getName();
+                }
+                return new MatchQuery(match.source(), fieldName, match.queryAsObject());
+            }
+
+            throw new IllegalArgumentException("Match must have a field attribute as the first argument");


Thank you @carlosdelest ! I'm on the fence about whether keeping these(defecting the real field name and throw IllegalArgumentException) on the data node or on the coordinator node(inside the Match function itself), so that we can keep MatchFunctionTranslator clean here, throw new IllegalArgumentException on data node caught my eyes. Is this something can be done safely on the coordinator node?

I have tried this, and the tests related to multi-typed fields that I can think of return as expected. I'd suggest to have more tests to cover more data types besides int and long with multi-type fields as part of the regression tests - double, unsigned_long, boolean, date, strings, without having them in the regression tests it is hard to tell when things change. semantic_text will be supported by match soon, depending on which PR merges first, we should have tests to cover it as well.

I'm on the fence about whether keeping these(defecting the real field name and throw IllegalArgumentException) on the data node or on the coordinator node(inside the Match function itself), so that we can keep MatchFunctionTranslator clean here, throw new IllegalArgumentException on data node caught my eyes. Is this something can be done safely on the coordinator node?

Hey @fang-xing-esql ! I added the IAE just to signal something broke in case we got there. We can't actually get to that point as this is already being validated in the validate() method for Match, which is being invoked at the coordinator level.

Having the IAE will help us notice in case the validate() method is not doing what it should. I can add some more info on the exception / comment if you think that's necessary.

. I'd suggest to have more tests to cover more data types besides int and long with multi-type fields as part of the regression tests - double, unsigned, boolean, strings, without having them in the regression tests it is hard to tell when things change

I understand. As I already created a conflicting mapping with employees, I will add some more tests to it.

@fang-xing-esql I added some tests in b182a87, LMKWYT!

I needed to fix some tests that depended on the basic mapping defaults in fa14ff5

…nt types

fang-xing-esql · 2024-12-04T21:15:23Z

Thanks for adding more tests @carlosdelest, the coverage looks much better. Besides the explicit casting on the matchQuery text, and error handling scenario below, I don't have other comments.

I came across one scenario when experimenting with multi-type fields, it is related to runtime error handling, it could be an edge case, it doesn't look very uncommon though. ES|QL and search handle them differently. In some situations, ES|QL handles errors softly, when runtime error happens to a particular record, it may not fail the entire query, it returns null for the record that has runtime error, and return normal results for the rest that don't have runtime errors. Search fails harder(in this particular case), if there is an issue with one record, the entire query fails. I'm not sure if we should make search functions' runtime error handling behave consistently with the other ES|QL functions in this PR, I just want to point this out, as I couldn't think of an easy way to make them behave consistently, you may have a better idea and decide the best option here.

schema
curl -u elastic:password -X PUT "localhost:9200/idx001?pretty" -H 'Content-Type: application/json' -d' 
{
  "mappings": {
    "properties": {
      "numericfield": {"type": "integer"},
      "stringfield": {"type": "text"},
      "mixedfield": {"type" : "integer"} 
    }
  }
}   
'      
curl -u elastic:password -X PUT "localhost:9200/idx002?pretty" -H 'Content-Type: application/json' -d'
{   
  "mappings": {
    "properties": {
      "numericfield": {"type": "long"},
      "stringfield": {"type": "keyword" },
      "mixedfield": {"type" : "keyword"} 
    }
  }
}
'
curl -u elastic:password -X PUT "localhost:9200/idx003?pretty" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "properties": {
      "numericfield": {"type": "double"},
      "stringfield": {"type": "text", "fields": {"keyword": {"type": "keyword"}}},
      "mixedfield": {"type" : "boolean"}
    }
  }
}
'
succeeded
+ curl -u elastic:password -v -X POST 'localhost:9200/_query?format=txt&pretty' -H 'Content-Type: application/json' '-d
{
  "query": "from idx00* | where match(stringfield::keyword, \"idx000\") | eval n = numericfield::double, s = stringfield::keyword, m = mixedfield::keyword | keep n, s, m | sort n"
}
'
       n       |       s        |        m         
---------------+----------------+------------------
[1.0, 4.0]     |[idx001, idx000]|[1, 4]            
[2.0, 4.0]     |[idx000, idx002]|[idx000, index002]
[3.0, 4.0]     |[idx000, idx003]|[false, true]     

failed
+ curl -u elastic:password -v -X POST 'localhost:9200/_query?format=txt&pretty' -H 'Content-Type: application/json' '-d
{
  "query": "from idx00* | where match(mixedfield::keyword, \"idx002\") | eval n = numericfield::double, s = stringfield::keyword, m = mixedfield::keyword | keep n, s, m | sort n"
}
'

{
  "error" : {
    "root_cause" : [
      {
        "type" : "query_shard_exception",
        "reason" : "failed to create query: For input string: \"idx002\"",
        "index_uuid" : "eIhp7cOqT8CYuO3aHkjqUg",
        "index" : "idx001",
        "suppressed" : [
          {
            "type" : "exception",
            "reason" : "1 further exceptions were dropped"
          },
          {
            "type" : "task_cancelled_exception",
            "reason" : "parent task was cancelled [cancelled on failure]"
          }
        ]
      }
    ],
    "type" : "query_shard_exception",
    "reason" : "failed to create query: For input string: \"idx002\"",
    "index_uuid" : "eIhp7cOqT8CYuO3aHkjqUg",
    "index" : "idx001",
    "caused_by" : {
      "type" : "number_format_exception",
      "reason" : "For input string: \"idx002\""
    },
    "suppressed" : [
      {
        "type" : "exception",
        "reason" : "1 further exceptions were dropped"
      },
      {
        "type" : "task_cancelled_exception",
        "reason" : "parent task was cancelled [cancelled on failure]"
      }
    ]
  },
  "status" : 400
}

+ curl -u elastic:password -v -X POST 'localhost:9200/_query?format=txt&pretty' -H 'Content-Type: application/json' '-d
{
  "query": "from idx00* | where match(mixedfield::double, 4.0) | eval n = numericfield::double, s = stringfield::keyword, m = mixedfield::double | keep n, s, m | sort n"
}
'

{
  "error" : {
    "root_cause" : [
      {
        "type" : "query_shard_exception",
        "reason" : "failed to create query: Can't parse boolean value [4.0], expected [true] or [false]",
        "index_uuid" : "yuxb5HGvTsOgsOA0o8nhvA",
        "index" : "idx003",
        "suppressed" : [
          {
            "type" : "exception",
            "reason" : "1 further exceptions were dropped"
          },
          {
            "type" : "task_cancelled_exception",
            "reason" : "parent task was cancelled [cancelled on failure]"
          }
        ]
      }
    ],
    "type" : "query_shard_exception",
    "reason" : "failed to create query: Can't parse boolean value [4.0], expected [true] or [false]",
    "index_uuid" : "yuxb5HGvTsOgsOA0o8nhvA",
    "index" : "idx003",
    "caused_by" : {
      "type" : "illegal_argument_exception",
      "reason" : "Can't parse boolean value [4.0], expected [true] or [false]"
    },
    "suppressed" : [
      {
        "type" : "exception",
        "reason" : "1 further exceptions were dropped"
      },
      {
        "type" : "task_cancelled_exception",
        "reason" : "parent task was cancelled [cancelled on failure]"
      }
    ]
  },
  "status" : 400
}

succeeded
+ curl -u elastic:password -v -X POST 'localhost:9200/_query?format=txt&pretty' -H 'Content-Type: application/json' '-d
{
  "query": "from idx00* | eval m = mixedfield::keyword | keep m"
}
'
        m         
------------------
index002          
[idx000, index002]
1                 
[1, 4]            
true              
[false, true]     

+ curl -u elastic:password -v -X POST 'localhost:9200/_query?format=txt&pretty' -H 'Content-Type: application/json' '-d
{
  "query": "from idx00* | eval m = mixedfield::double  | keep m "
}
'
       m       
---------------
1.0            
[0.0, 1.0]     
1.0            
[1.0, 4.0]     
null           
null

…s provided in the query

carlosdelest · 2024-12-05T21:17:08Z

I came across one scenario when experimenting with multi-type fields, it is related to runtime error handling

@fang-xing-esql , that is due to match query behaviour. The good news, it can be controlled by the lenient parameter, precisely for this purpose.

I've added leniency to match queries and a test for this behaviour in 9060cba .

Besides the explicit casting on the matchQuery text

I will remove it on Monday if @ioanatia doesn't have any comments to that 👍

# Conflicts: # x-pack/plugin/esql/qa/testFixtures/src/main/java/org/elasticsearch/xpack/esql/EsqlTestUtils.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/PushFiltersToSource.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/planner/EsqlExpressionTranslators.java # x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/optimizer/LocalPhysicalPlanOptimizerTests.java

carlosdelest · 2024-12-09T11:37:18Z

@fang-xing-esql I removed the possibility to cast the query to a specific type in e723ae6 .

I think this and the leniency work above should address your remaining concerns - LMKWYT!

fang-xing-esql

Thank you @carlosdelest ! LGTM.

elasticsearchmachine · 2024-12-09T18:57:25Z

💔 Backport failed

Status	Branch	Result
❌	8.x	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 117555

carlosdelest · 2024-12-09T19:41:49Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Questions ?

Please refer to the Backport tool documentation

…#117555) (#118297) * Fix and unmute synonyms tests using timeout (#117486) (cherry picked from commit 930a99c) # Conflicts: # muted-tests.yml # rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/synonyms/10_synonyms_put.yml # rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/synonyms/110_synonyms_invalid.yml # rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/synonyms/20_synonyms_get.yml # rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/synonyms/30_synonyms_delete.yml # rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/synonyms/40_synonyms_sets_get.yml # rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/synonyms/50_synonym_rule_put.yml # rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/synonyms/60_synonym_rule_get.yml # rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/synonyms/70_synonym_rule_delete.yml # rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/synonyms/80_synonyms_from_index.yml # rest-api-spec/src/yamlRestTest/resources/rest-api-spec/test/synonyms/90_synonyms_reloading_for_synset.yml * Fix merge

carlosdelest added 12 commits November 26, 2024 10:42

Add support for match types in match function

34c7cbb

Add tests

e260d68

Use queryAsObject() instead of queryAsString() to provide match query…

0395751

… with the appropriate object

Changed function signature docs

f901033

Add CSV tests for types

b4c8c52

Add match operator CSV tests

ca600bb

Merge branch 'main' into non-issue/esql-match-type-compat

2a1d6f2

# Conflicts: # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/optimizer/rules/physical/local/PushFiltersToSource.java

Fix merge

35193e3

Spotless

f1e7ec0

Fix merge

005d3b3

Fix tests

caba4ba

Add types to match operator tests and docs

bd846b9

elasticsearchmachine added the v9.0.0 label Nov 26, 2024

carlosdelest commented Nov 26, 2024

View reviewed changes

carlosdelest changed the title ~~ESQL: Match function / operator expand type compatibility~~ ESQL: Expand type compatibility for match function and operator Nov 26, 2024

carlosdelest added auto-backport Automatically create backport pull requests when merged v8.18.0 labels Nov 26, 2024

carlosdelest mentioned this pull request Nov 22, 2024

ES|QL MATCH roadmap #116261

Open

carlosdelest added 6 commits November 26, 2024 13:36

Fix non-integer field test

f2ba267

Remove compatibility test that no longer works

5f0f502

Add new capability for BwC tests

33c069e

Fix forbidden API

7561f58

Add capability to tests

96583d7

Fix tests

22dcdb4

carlosdelest requested review from tteofili, afoucret, ioanatia and ChrisHegarty November 27, 2024 06:45

carlosdelest and others added 3 commits November 29, 2024 10:49

Update docs/changelog/117555.yaml

cf3f705

Fix changelog

8102163

Fix tests ignoring order

edf9838

ioanatia reviewed Dec 2, 2024

View reviewed changes

ioanatia approved these changes Dec 2, 2024

View reviewed changes

fang-xing-esql reviewed Dec 3, 2024

View reviewed changes

Adding support for casting on the field type and allowing multi-index…

347b3ce

… queries with different types

fang-xing-esql reviewed Dec 3, 2024

View reviewed changes

carlosdelest added 2 commits December 4, 2024 09:57

Additional tests and checks for using multi index fields with differe…

b182a87

…nt types

Merge branch 'main' into non-issue/esql-match-type-compat

d8f5e0b

carlosdelest requested a review from fang-xing-esql December 4, 2024 08:59

Fixed tests to include new field type

fa14ff5

Use lenient match queries to allow mixed data fields with wrong value…

9060cba

…s provided in the query

carlosdelest added 4 commits December 9, 2024 11:49

Remove casting on match query side

e723ae6

Fix merge

b0ac5f1

Spotless

71e78f9

fang-xing-esql approved these changes Dec 9, 2024

View reviewed changes

carlosdelest merged commit eb59b98 into elastic:main Dec 9, 2024
16 checks passed

elasticsearchmachine added the backport pending label Dec 9, 2024

carlosdelest mentioned this pull request Dec 9, 2024

[8.x] ESQL: Expand type compatibility for match function and operator (#117555) #118297

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ESQL: Expand type compatibility for match function and operator #117555

ESQL: Expand type compatibility for match function and operator #117555

carlosdelest commented Nov 26, 2024 •

edited

Loading

github-actions bot commented Nov 26, 2024

carlosdelest Nov 26, 2024

carlosdelest Nov 26, 2024

carlosdelest Nov 26, 2024

carlosdelest Nov 26, 2024

ChrisHegarty Nov 27, 2024

carlosdelest Nov 29, 2024

carlosdelest Nov 26, 2024

carlosdelest commented Nov 27, 2024

ioanatia Dec 2, 2024

carlosdelest Dec 2, 2024

ioanatia Dec 2, 2024

carlosdelest Dec 2, 2024

fang-xing-esql left a comment •

edited

Loading

fang-xing-esql Dec 2, 2024 •

edited

Loading

carlosdelest commented Dec 3, 2024

fang-xing-esql commented Dec 3, 2024

carlosdelest commented Dec 3, 2024

fang-xing-esql Dec 3, 2024 •

edited

Loading

carlosdelest Dec 3, 2024

carlosdelest Dec 4, 2024 •

edited

Loading

fang-xing-esql commented Dec 4, 2024

carlosdelest commented Dec 5, 2024

carlosdelest commented Dec 9, 2024

fang-xing-esql left a comment

elasticsearchmachine commented Dec 9, 2024

carlosdelest commented Dec 9, 2024

ESQL: Expand type compatibility for match function and operator #117555

ESQL: Expand type compatibility for match function and operator #117555

Conversation

carlosdelest commented Nov 26, 2024 • edited Loading

github-actions bot commented Nov 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carlosdelest commented Nov 27, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fang-xing-esql left a comment • edited Loading

Choose a reason for hiding this comment

fang-xing-esql Dec 2, 2024 • edited Loading

Choose a reason for hiding this comment

carlosdelest commented Dec 3, 2024

fang-xing-esql commented Dec 3, 2024

carlosdelest commented Dec 3, 2024

fang-xing-esql Dec 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carlosdelest Dec 4, 2024 • edited Loading

Choose a reason for hiding this comment

fang-xing-esql commented Dec 4, 2024

carlosdelest commented Dec 5, 2024

carlosdelest commented Dec 9, 2024

fang-xing-esql left a comment

Choose a reason for hiding this comment

elasticsearchmachine commented Dec 9, 2024

💔 Backport failed

carlosdelest commented Dec 9, 2024

💚 All backports created successfully

Questions ?

carlosdelest commented Nov 26, 2024 •

edited

Loading

fang-xing-esql left a comment •

edited

Loading

fang-xing-esql Dec 2, 2024 •

edited

Loading

fang-xing-esql Dec 3, 2024 •

edited

Loading

carlosdelest Dec 4, 2024 •

edited

Loading